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The  Presidio  of  Monterey  Field  Unit  is  recognized  for  its  applied  research 
in  unit  training  and  evaluation.  Past  research  and  development  has  emphasized 
test  and  evaluation  of  emerging  training  systems,  training  methodology  for 
tactical  engagement  simulation,  and  innovative  approaches  to  solving  problems 
in  training  management. 

Improving  maneuver-arms  tactical  training  is  a  high  priority  for  the  Army. 
ARI  has  developed  strategies,  techniques,  and  practical  guidance  in  support  of 
the  Army  Training  and  Evaluation  Program  (ARTEP).  This  research  product,  one  of 
several  recent  efforts  along  this  line,  is  designed  to  provide  evaluation  teams 
with  step-by-step  guidance  on  how  to  prepare  for  and  conduct  evaluations  of 
maneuver-arms  tactical  field  exercises. 
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HOW  TO  EVALUATE  UNIT  PERFORMANCE 
INTRODUCTION 

The  purpose  of  this  document  is  to  provide  guidance  on  preparing  and 
conducting  evaluations  of  maneuver-arms  unit  performance. 

The  guidance  presented  below  is  generally  applicable  to  formal  (external) 
and  informal  (internal)  evaluations.  Both  types  of  evaluations  are  treated  as 
stemming  from  training  exercises.  That  is,  training  is  a  major  purpose  of  the 
exercise — it  is  not  conducted  only  to  evaluate  the  unit.  Thus,  note  taking, 
debriefings.  After  Action  Reviews,  etc.  are  included  as  basic  ingredients  of 
evaluation  procedures.  This  document  contains  guidance  for  evaluations 
conducted  ;.o  (a)  provide  units  with  diagnostic  feedback  on  their  performance, 
and  (b)  to  provide  higher  echelons  with  information  on  the  units'  training 
needs  and  proficiency. 

This  document  has  several  other  characteristics  which  will  affect  its 
use.  First,  it  is  expressly  designed  to  provide  guidance  on  evaluations  based 
on  unit  tactical  field  exercises.  Therefore,  some  of  the  guidance  will  not  be 
needed  for  evaluating  TWETs,  CPXs,  MAPEXs,  etc.  Second,  some  of  the  guidance 
is  specific  to  tactical  engagement  simulation  (MILES)  exercises.  For 
exercises  which  do  not  require  an  OPFOR  and  MILES  (e.g..  Tactical  Road  March), 
the  material  dealing  with  the  MILES-related  procedures  can  be  ignored.  Third, 
there  arc  a  variety  o'  actions  that  indirectly  support  and  affect  the  quality 
of  training  evaluations  which  will  not  be  covered.  Some  examples  ate 
selection  of  suitable  terrain,  scenario  development,  establishment  of 
appropriate  force  ratios  and  decisions  concerning  employment  of  OPFOR 
firepower.  Finally,  this  guidance  assumes  that  there  is  no  shortage  of 
required  training  resources,  mainly  because  the  types  and  extents  of  shortages 
vary  considerably  from  one  location  to  another.  Thus,  the  guidance  provided 
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here  is  for  the  "ideal  case."  Adjustments  to  fit  local  circumstances  will 
probably  be  required. 

There  are  two  basic  audiences  for  this  document:  those  in  charge  of  the 
evaluation  and  the  evaluators.  Most  of  the  material  presented  is  directed  to 
the  officer(s)  responsible  for  preparing  for  and  insuring  the  quality  of  the 
evaluation.  Although  the  evaluation  team  members  could  profit  from  careful 
reading  of  the  following  material,  their  attention  should  be  directed 
primarily  to  STAGE  3:  "During  the  Exercise,”  and  STAGE  4:  "After  Exercise.” 

Although  limited  information  on  exercise  control  requirements  are  covered 
in  the  document,  detailed  explanations  are  presented  in  FM  25-5,  How  to  Plan, 
Prepare,  and  Conduct  Tactical  Training  with  MILES. 

The  evaluation  guidance  is  broken  into  five  stages  (Figure  1)  and  each 
stage  consists  of  a  series  of  steps.  The  title  of  each  stage  indicates  when 
the  steps  are  to  be  carried  out,  and  the  title  of  each  step  indicates  what  is 
to  be  done.  The  first  stage  includes  those  planning  and  preparation  steps 
which  should  be  accomplished  before  the  unit  moves  to  the  field  for  the 
exercise.  The  second  stage  includes  final  preparation  steps  which  are  carried 
out  immediately  before  the  start  of  the  exercise.  The  third  stage  includes 
the  operations  carried  out  during  the  tactical  exercise.  The  fourth  stage 
consists  of  the  steps  to  be  accomplished  in  preparing  and  conducting  after 
action  reviews  (AARs).  The  fifth  and  final  stage  consists  of  those  steps  to 
be  accomplished  in  preparing  an  after  action  report  or  briefing. 
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STAGES 


STEPS 
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STAGE  I.  "In  Garrison"  consists  of  four  steps: 
STEP  1.  Review  Exercise  Plans 
STEP  2.  Select  Evaluators 
STEP  3.  Brief  Evaluators 
STEP  4.  Train  Evaluators 


STEP  1. 
_ 


Review  Exercise  Plans 


\ 


1.  The  evaluation  team  leader  should  be  familiar  with  the  purpose,  scope 
and  general  requirements  of  the  evaluation  exercise  before  any  other  actions 
are  begun.  For  formal  (external)  evaluations  the  necessary  information  will 
usually  be  contained  in  the  tasking  documents.  For  informal  (internal) 
evaluations  the  necessary  information  may  be  developed  in  battalion  staff 
meetings,  by  the  battalion  S-3,  or  b'r  the  company  commander.  Issues  of 
concern  in  the  initial  review  include: 

a.  High  priority  training  objectives. 

b.  Tactical  scenario  and  terrain. 

c.  Time  available  for  preparation. 

d.  Special  equipment  needed. 

e.  Evaluation  team  manpower  requirements* 

f.  Coordination  and  administrative  requirements. 

2.  Prepare  an  information  package  for  the  evaluation  team  members. 
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Include  as  many  details  of  the  evaluation  operation  as  possible.  An  example 
of  what  such  a  package  might  contain  is  provided  in  Table  1  (page  8).  To  the 
extent  that  details  are  not  known  at  this  early  stage,  plan  to  acquire  the 
information  before  the  evaluation  team  briefing  (STEP  3).  For  less  structured 
evaluations,  usually  internal  evaluations,  information  may  be  provided  in  the 
team  briefing  without  written  materials. 


STEP  2.  Seise L  evaluators 


) 


The  quality  of  the  evaluation  is  dependent  on  the  experience  and  expertise 
of  the  evaluators.  Every  effort  should  be  made  to  select  the  best  quality 
personnel  available  to  serve  as  evaluators.  Some  considerations  are  as 
follows: 

1.  RANK.  The  evaluators'  ranks  should  be  one  higher  than  the 
highest  rank  in  the  echelon  being  evaluated.  This  structure  tends  to  minimize 
disputes  and  to  make  operations  smoother. 

2.  EVALUATION  EXPERIENCE.  Evaluators  should  be  selected  who  have 
previously  served  as  evaluators,  and  whose  performance  in  that  role  has  been 
found  acceptable.  This  will  tend  to  minimize  (a)  the  amount  of  evaluator 
training  required,  (b)  the  amount  of  time  required  to  prepare  for  the 
exercise,  and  (c)  the  amount  of  supervision  needed  during  the  evaluation 
process.  If  evaluators  who  have  little  or  no  previous  experience  must  be 
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used,  they  should  be  paired  with  experienced  evaluators  during  the  evaluator 


training  period.  Experienced  evaluators  should  be  assigned  partial 

responsibility  for  the  training  of  their  less  experienced  counterparts.  Such 
pairing  tends  to  reduce  the  amount  of  time  and  effort  required  for  training 
and  facilitates  the  hand-off  of  methods,  techniques,  and  lessons  learned  by 
the  experienced  evaluators.  Whenever  possible  inexperienced  evaluators  should 
accompany  experienced  evaluators  during  an  exercise  prior  to  assuming 
responsibility  for  evaluating  a  unit.  Such  "hands-on"  evaluator  training 
usually  results  in  improved  quality  of  the  evaluation. 

3.  LEADER  EXPERIENCE.  Evaluators  should  be  selected  who  are 
currently,  or  have  recently  been,  leaders  of  the  echelon  they  will  be 
evaluating.  Many  training  objectives  require  substantial  leader  experience  to 
interpret  and  apply  accurately.  Using  experienced  leaders  can  also  reduce  the 
amount  of  training  needed.  Although  it  is  extremely  rare  to  have  them 
available,  leaders  with  combat  experience  should  be  assigned  to  key  evaluator 
positions  whenever  possible. 

4.  TO&S.  Evaluators  should  be  selected  who  have  had  recent 
assignments  in  '-tie  type  of  TO&E  unit  to  be  evaluated  and  should  have  an 
OSC/MOS  appropriate  to  the  evaluated  unit's  TO&E. 

5.  NUMBER  OF  EVALUATORS.  The  number  of  evaluators  required 
depends  on  the  purpose  of  the  evaluation.  However,  in  general,  enough 
evaluators  should  be  selected  to  cover  at  least  one  and  preferably  two 
echelons  below  the  one  to  be  evaluated,  plus  at  least  one  evaluator  for  the 
OPFOR,  one  for  each  specialized  attached  unit  or  section  (e.g.,  FIST),  and  one 
senior  evaluator.  In  addition  a  few  alternates  should  be  selected  to  cover 
for  unexpected  absences,  or  to  serve  as  "trainee”  evaluators  to  be  used  In 
future  evaluations.  Alternate  evaluators  can  fill  desirable  functions  which 
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tactical  radio  net  monitor) 


are  not  strictly  required  (e.g.. 


STEP  3.  Brief  the  Evaluators 


1.  Begin  the  briefing  by  handing  out  an  evaluator  information 
package.  An  example  of  the  contents  of  such  a  package  is  provided  in  Table 
1.  Try  to  make  the  package  as  complete  as  possible  and  have  its  sections 
arranged  in  the  order  in  which  you  intend  to  brief  them. 

2.  PURPOSE  AND  SCOPE  OF  THE  EVALUATION.  Briefly  explain  why  the 
evaluation  exercise  is  being  conducted  and  what  the  evaluation  team  will  be 
expected  to  accomplish.  Describe  the  unit(s)  to  be  evaluated,  and  note  any 
special  characteristics  which  may  affect  the  evaluation  process  (e.g.,  under 
strength). 

3.  OVERVIEW  OF  THE  TACTICAL  SCENARIO.  Provide  a  brief  description 
of  the  scenario  and  the  key  tasks  and  sub-tasks  which  are  to  be  evaluated. 
Provide  just  enough  detail  to  give  the  evaluators  a  concept  of  the 
operations.  Do  not  overwhelm  them  with  details.  Details  can  better  be 
presented  during  the  evaluator  training  period  (STEP  4). 

4.  OVERALL  SCHEDULE  OF  EVENTS.  This  should  include  all  major 
events  from  evaluator  training  through  preparation  of  the  after  action  report 
(if  required).  Although  the  schedule  need  not  include  precise  dates  and 
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times,  it  should  indicate  the  sequence  in  which  events  are  expected  to 
occur.  The  objective  here  is  to  give  the  evaluators  some  idea  of  what  is 

Table  l 

Example  of  Evaluator's  Information  Package  Contents 

a.  Statement  of  Purpose  and  Scope 

b.  Tactical  Scenario  Description  (with  maps  and  overlays, 

i  as  required) 

i 

c.  ARTEP  Training  and  Evaluation  Outlines,  or  Evaluator's 
Test  Questions  (Army  Unit  Tests) 

i 

d.  Overall  Schedule  and  Exercise  Schedule  i 

i 

e.  Evaluator  Training  Plans 

» 

f.  Evaluator  Training  Support  Materials  1 

i 

g.  Control  Plan  (including  Evaluation  Team  Organization)  i 

h.  Communication  Plan 

i.  Administrative  and  Logistic  Details 

t 

!  j.  Safety  Instructions 


k.  Transportation 


supposed  Lo  happen.  Naturally,  if  a  reliable  schedule  has  been  developed  in 
detail,  it  can  be  provided  in  the  information  package.  The  evaluators  should 
be  told  to  see  the  OIC/NCOIC  at  the  end  of  the  briefing  if  they  have  any 
conflicts  with  the  schedule  of  events. 

5.  EVALUATOR  TEAM  TASK  ORGANIZATION.  The  task  organization  is 
usually  easiest  to  present  as  a  block  diagram.  Often  the  organization  can  be 
shown  as  an  overlay  of  Lhe  organization  of  the  unit  being  evaluated,  plus  the 
OPFOR  evaluator(s)c 

6.  EVALUATOR  TRAINING  PLANS.  Provide  an  overview  of  Lhe  training 
to  be  given  to  the  evaluation  team,  including  MILES  controller  training.  Each 
topic  to  be  covered  during  the  evaluator  training  period  should  be  introduced. 

7.  EVALUATION  TEAM  MATERIALS.  Evaluation  team  members  will  need 
some  special  items.  A  brief  list  is  provided  in  Table  2  (p.  10). 


Stage  Step 


1.  By  the  end  of  the  evaluator  training  period  evaluators  should 
have  thorough  knowledge  of  the  following: 

a.  Evaluation  Performance  Criteria. 

b.  Tactical  Scenario  and  Terrain. 

c.  Exercise  Control  Plan. 
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Table  2 


Example  of  Evaluators'  Materials 


Map 

Compass  (optional) 

MILES  Controller  Gun,  Controller  Key  and  Controller 
Handbook 

Distinctive  Clothing  (evaluators  should  be  dressed 
so  that  they  are  easily  distinguished  from  the  playe 
from  a  distance) 

Clipboard  and  Pencils 
Watch 

Radio  (for  exercises  at  or  above  platoon  level) 
Binoculars  (for  exercises  using  large  lanes) 


d.  Training  Aids  and  Devices  (e.g.,  MILES). 

e.  Administrative,  Logistic  and  Safety  Procedures. 

f.  How  to  lead  an  After  Action  Review  (AAR). 

2.  There  are  a  number  of  training  methods  which  can  be  used  to 
achieve  these  objectives.  These  include: 

a.  Class  Instruction. 

b.  Evaluator  Workbooks. 

c.  Observation  and  Discussion  of  Tactical  Exercises. 

d.  Terrain  Reconnaissance. 

3.  CLASS  INSTRUCTION  . 

a.  The  class  may  be  conducted  in  a  classroom  or  in  Lhe 
field.  If  the  evaluation  team  is  able  to  go  to  the  field  for  preliminary 
terrain  reconnaissance  the  class  should  be  held  just  prior  to  the 
reconnaissance. 

The  instructor  should  make  the  class  as  interactive  as  possible. 
That  is,  the  instructor  should  guide  the  class  by  asking  questions  and  having 
the  class  answer  them.  This  approach  tends  to  hold  the  attention  of  the  class 
better  than  a  lecture  format. 

The  most  important  topics  to  be  covered  in  the  class  are  the  T&EOs 
(or  Evaluator  Test  Questions  (ETQ)  for  Army  Unit  Tests)  and  the  tactical 
scenario.  As  you  present  each  part  of  the  Tactical  Scenario,  discuss  each 
task,  subtask,  condition,  and  standard.  Ask  the  evaluators  to  define 
important  tactical  terms  in  order  to  insure  that  they  are  all  working  from  the 
same  definitions.  Some  evaluation  standards  are  written  to  allow  for 
flexibility  in  application.  Terms  like  "immediate,’*  "appropriate,”  etc.  will 
require  working  definitions  in  order  to  help  apply  them  in  a  consistent  way. 
These  should  be  supplied  by  the  senior  evaluator.  Extensive  group  discussions 


of  these  kinds  of  definitions  should  be  avoided.  Each  evaluator  should  have  a 
map  of  the  exercise  lane  and  a  copy  of  the  appropriate  T&EO.  They  should 
follow  along  using  both  of  these  as  the  instructor  discusses  the  exercise. 
The  class  should  be  led  to  visualize  the  exercise  developing.  For  each  T&EO 
standard  ask  the  class:  "How  would  you  know  if  the  unit  were  meeting  the 
standard?**  Ask  the  class  to  give  examples.  Have  them  specify  observable 
behaviors  which  indicate  success  or  failure  on  each  standard  or  question.  A 
number  of  problems  commonly  encountered  in  applying  ARTEP  T&EO  standards  are 
discussed  in  Stage  III,  Step  1  (p.  18).  At  a  minimum  these  classes  of 

problems  should  be  covered  with  the  evaluation  team. 

The  main  discussion  will  naturally  focus  on  the  unit  to  be 
evaluated.  However,  the  OPFOR  plans  and  role  in  the  scenario  also  should  be 
discussed  in  detail. 

b.  Discuss  the  Exercise  Control  Plan.  The  discussion  should 

cover: 

(1)  Evaluation  Team  Chain  of  Command. 

(2)  Comm -nications  (Frequencies  and  Call  Signs  can  be 
provided  in  handouts,  though  it  is  usually  best  to  provide  this  information 
after  going  to  the  field  (STAGE  II). 

(a)  Tactical  Nets  (Unit  and  OPFOR). 

(b)  Evaluation  and  Control  Net. 

(c)  Administrative  Net. 

(3)  Control  Measures  (lane  boundaries,  checkpoints,  phase 

lines,  etc.). 

(4)  Responsibilities  of  each  Lype  of  evaluator  position. 

(5)  kules  of  Engagement  (this  is  especially  critical  in 

MILES  exercises). 
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(6)  Control  Procedures. 

c.  Discuss  Administrative,  Logistic  and  Safety  Procedures. 

d.  Determine  how  many  and  which  evaluators  have  had  MILES 
training.  If  most  evaluators  have  had  MILES  training,  pair  those  who  have  not 
with  those  that  have.  If  this  is  not  feasible,  plan  to  allocate  the  non-MILES 
trained  evaluators  to  the  least  critical  positions. 

If  few  evaluators  have  had  MILES  training,  arrange  for  at  least  two  days  of 
training.  If  this  is  not  feasible  the  evaluation  will  suffer.  The  quality  of 
the  evaluation  is  dependent  on  the  quality  of  exercise  control,  which  is  in 
turn  dependent  on  detailed  knowledge  of  the  capabilities  and  requirements 
specific  to  MILES  exercises. 

e.  At  the  end  of  the  class  instruction  you  should  have 
identified  areas  in  which  the  evaluation  team’s  knowledge  needs  to  be 
improved.  Assign  the  team  some  reading,  if  necessary,  to  correct  Lhe  major 
shortfalls.  But  remember,  do  not  overload  the  team  with  material.  Be 
specific.  Do  not  assign  entire  volumes  but  rather  identify  specific  pages  or 
sections.  Plan  to  ask  questions  on  the  assigned  material  over  the  next  day  or 
so,  and  tell  the  evaluators  that  you  intend  to  do  so. 

f.  The  class  discussions  will  give  you  a  reasonably  good  idea 
of  their  competency  and  may  reveal  some  problems  not  previously  recognized. 
Also,  the  evaluation  team  discussions  may  give  you  a  good  basic?  for  assigning 
team  members  to  positions  for  the  exercise. 

4.  EVALUATOR  WORKBOOKS.  A  self-paced  evaluator  workbook  has  been 
developed  and  is  available  in  the  ARTEP  Mission  Training  Plan.  The  workbook 
can  minimize  the  amount  of  time  the  evaluation  team  leader  needs  to  spend 
training  the  evaluation  team.  The  workbook  covers  the  evaluators'  duties  from 
the  planning  stage  through  the  After  Action  Review.  The  workbook  should  not 
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be  used  as  the  sole  training  method  but  rather  should  be  used  in  conjunction 
with  other  methods  described  in  this  section. 

5.  OBSERVATION  AND  DISCUSSION  OF  TACTICAL  EXERCISES.  If  possible, 
arrange  for  the  evaluation  team  to  observe  tactical  exercises  of  the  type  they 
will  be  evaluating.  Follow  the  observation  by  a  discussion  of  the  T&EOs  and 
tactical  scenario.  The  team's  discussion  should  focus  on  how  to  apply  T&EO 
standards  and  techniques  for  observing  exercises. 

6.  PRELIMINARY  TERRAIN  RECONNAISSANCE..  Take  the  evaluation  team 
to  the  site  of  the  upcoming  exercise.  Walk  or  drive  the  lane(s)  completely, 
and  follow  the  tactical  scenario  and  ARTEP  T&EO  through  each  stage  of  the 
evaluation.  The  evaluation  team  should  be  familiar  with: 

a.  the  probable  routes  of  advance; 

b.  probable  defensive  positions  (main,  forward  and  secondary 

positions); 

c.  OPFOR  and  administrative  control  measures; 

d.  good  sites  for  observing  the  exercise; 

e.  location  of  administrative  HQ; 

f.  probable  locations  of  the  first,  and  major  later 

engagements; 

g.  probable  defensive  "blind  spots"  or  "dead  zones"; 

h.  highly  vulnerable  parts  of  offensive  movement  routes. 
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STAGE  II.  "Just  Before  the  Exercise"  consists  of  three  steps: 
STEP  1.  Assign  Evaluators 
STEP  2.  Brief  Evaluators 
STEP  3.  Final  Lane  Reconnaissance 


c 


STEP  1.  Assign  Evaluators 


J 


Stage 

Step 

■ 
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■ 

cover 

the  most 

difficult  to  evaluate  elements  in  the  exercise. 

2.  The  senior  evaluator  should  cover  the  leader  of  the  evaluated 

unit. 


3.  At  least  one  evaluator  should  observe  the  exercise  from  the 
OPFOR  perspective. 

4.  Each  evaluator  should  know  who  is  immediately  above  and  below 
him  in  the  evaluation  team's  chain  of  command. 

5.  If  extra  evaluators  are  available,  assign  one  to  listen 
exclusively  to  the  tactical  radio  net  and  make  notes  on  engagements  (times, 
locations,  etc.)  SITREPs,  and  other  potentially  significant  events. 

6.  Inexperienced  evaluators  should  be  paired  with  experienced  ones 
whenever  possible. 

7.  Evaluators  who  are  least  qualified  can  be  assigned  to  pass  put, 
collect,  and  account  for  data  forms,  evaluator  team  equipment,  etc. 
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8.  After  evaluators  have  been  assigned,  pass  out  the  evaluation 


forms  (T&EOs  or  ETQs),  and  other  required  items. 


^  STEP  2. Brief  Evaluators 


This  is  the  last  opportunity  to  brief  the  evaluators  before  the 
exercise  begins.  All  final  instructions  to  the  team  should  be  given. 

1.  Review  the  training  objectives. 

2.  Review  the  rules  of  engagement. 

3.  Review  Exercise  Control  Measures.  Include  the  control  measures 
for  the  tactical  unit  if  known.  Evaluators  should  enter  control  measures  on 
their  maps. 

4.  Provide  list  of  call  signs  and  frequencies. 

a.  Exercise  Control  Net. 

b.  Tactical  Nets  (Unit  and  OPFOR). 

c.  Administrative  Net. 

5.  Briefly  review  administrative,  log  .stic  and  safety  procedures 
and  instructions. 
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STEP  3.  Final  Lane  Reconnaissance 


Stage  Step 

|_rS 

■  S 


1.  The  evaluation  team  should  already  be  familiar  with  the  terrain 
from  the  preliminary  reconnaissance  (STAGE  I,  STEP  4).  The  final 
reconnaissance  will  focus  on  the  OPFOR  positions  or  routes.  This  will  help 
the  evaluation  team  to  anticipate  the  evaluated  unit's  actions  and  responses 
to  the  OPFOR,  and  will  improve  the  quality  of  the  evaluation. 

2.  If  the  OPFOR  is  in  the  defense: 

a.  Inspect  the  major  weapon  system  positions. 

b.  Inspect  forward,  main  and  secondary  defensive  positions. 

c.  Have  OPFOR  leader  brief  the  evaluation  team  on  the 
defensive  plan,  including  the  fire  support  plan. 

d.  Determine  the  OPFOR  fields  of  observation  and  fire. 

e.  Identify  "dead  zones”  (places  which  cannot  be  covered  by 
direct  fire). 

f.  Identify  both  best  and  probable  routes  into  the  OPFOR 
position(s). 

g.  Inspect  counter-mobility  measures  (minefields,  obstacles, 
etc. ) 

3.  If  the  OPFOR  is  in  the  offense: 

a.  Inspect  the  route  of  advance  from  the  point  at  which  the 
unit  is  expected  to  detect  the  OPFOR  approach. 
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b.  Have  the  OPFOR  leader  brief  the  evaluation  team  on  his  plan 
of  attack,  including  his  fire  support  plan. 

c.  Identify  probable  locations  of  initial  engagements. 

d.  Identify  locations  along  the  planned  OPFOR  route  at  which 
they  are  particularly  vulnerable  to  direct  fire. 

4.  Finally,  the  evaluation  team  should  carry  out  a  communication 
check. 


STAGE  III.  "During  the  Exercise"  consists  of  two  steps: 
STEP  1.  Collect  Evaluation  Data 
STEP  2.  Terminate  Exercise 


STEP  1.  Collect  Evaluation  Data 


Stage  Step 

■  r? 


Illh 
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1.  During  the  exercise,  the  evaluation  team  will  need  to  collect 
two  types  of  data:  a)  notes  on  important  exercise  events  based  on  the 
evaluation  team's  observations  and  b)  responses  to  ARTEP  T&EO  standards  (or 
for  Army  Unit  Tests,  Evaluator's  Test  Questions).  Notes  will  be  used  after 
the  exercise  to  clarify  and  to  add  detail  to  the  data  from  T&EO  standards.  In 
the  following,  [-aragraphs  2  through  7  cover  observation  and  note  taking,  and 
paragraphs  8  and  9  cover  the  application  of  T&EO  standards. 
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2.  Each  evaluator  should  make  notes  on  important  events  during  the 
exercise.  Make  notes  on  the  evaluation  forms.  Write  down  the  time  each 
important  event  occurs.  Notes  should  include  "what,  when,  to  whom,  and  how” 
each  event  occurred. 

3.  Evaluators  should  pay  close  attention  to  the  orders  given  by 
the  unit  leader.  Make  notes  on  key  aspects  of  the  order,  and  on  significant 
omissions. 

4.  If  unit  is  in  the  defense: 

a.  Inspect  the  positions. 

b.  Determine  fields  of  observation  and  fire. 

c.  Identify  "dead  zones." 

d.  Identify  probable  OPFOR  routes  into  the  position. 

e.  Identify  elements  most  likely  to  make  initial  contact,  and 
select  a  position  allowing  good  observation. 

f.  The  evaluator  with  the  OPFOR  should  inform  the  evaluation 
team  when  the  OPFOR  erf  sses  their  line  of  departure. 

5.  If  the  unit  is  on  the  offense: 

a.  Move  with  or  near  the  unit. 

b.  Move  so  that  you  will  be  in  a  good  position  to  observe 
initial  contact. 

c.  If  a  unit  splits  up  the  senior  evaluator  should  be  inTormed 

immediately. 

d.  Usually,  it  is  best  to  move  with  the  forward-most  elements 
of  the  unit.  The  next  higher  evaluator  should  try  to  insure  that  the  rear 
elements  are  covered.  This  is  especialy  important  when  the  lead  unit  splits 
up,  or  when  elements  of  a  unit  are  likely  to  come  in  contact  with  the  OPFOR. 

e.  As  your  uni.  passes  or  crosses  administrative  or  tactical 
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control  measures,  inform  the  next  higher  evaluator  immediately. 

6.  Do.  not  compromise  locations,  firing  positions  or  movement 
routes  of  the  unit  or  of  the  OPFOR.  Remain  tactical  throughout  the  exercise. 

7.  Keep  in  contact  with  evaluators  of  adjacent  units. 

8.  The  basic  idea  of  the  T&EO  is  simple.  The  evaluator  determines 
whether  or  not  a  unit  met  the  performance  standard  for  a  given  task,  performed 
under  certain  conditions.  On  the  evaluation  form,  the  evaluator  indicates 
whether  standards  were  achieved,  not  achieved,  or  were  not  observed  or 
evaluated. 

9.  However,  there  are  some  complications  in  applying  performance 
standards.  The  following  are  commonly  encountered  problems  and  suggestions  on 
what  to  do  about  them. 

a.  Seme  standards  refer  to  actions  which  occur  repetitively. 
This  situation  occurs  very  often.  The  evaluator's  problem  is  how  to  evaluate 
the  performance  if  sometimes  the  unit  meets  the  standard  and  sometimes  it 
doesn't.  _ _  _ 

EXAMPLES:  "MOVEMENT  IS  CONTROLLED  BY  VISUAL  SIGNALS  WHEN  OUT  OF 
CONTACT"  (ARTEP  71-2,  3-IV-6-6) 

"THE  MANEUVER  ELEMENT  BOUNDS  TO  THE  SUBSEQUENT  OVERWATCH 
POSITION  EMPLOYING  THE  APPROPRIATE  MOVEMENT  TECHNIQUE 
.  INTERNALLY."  (ARTEP  71-2,  3-IV-6) 


Such  actions  are  often  appropriate  many  times  or  continuously 
during  a  single  exercise;  but  the  evaluator  is  to  give  a  single  response. 

SUGGESTIONS:  Use  the  70  percent  rule.  That  is,  for  any  standard 
which  is  applicable  continuously  or  several  times  during  an  exercise,  if  the 
unit  performs  the  required  action  at  least  70  percent  of  Lhe  time,  indicate 
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that  the  standard  has  been  met.  To  help  you  do  this,  keep  a  tally  on  your 
evaluation  form  of  the  number  of  “GO**  and  “NO  GO"  cases.  Make  notes  on 
unusually  important  cases.  For  standards  which  are  appropriate  throughout  the 
exercise  look  for  the  behavior  every  5  to  10  minutes.  Try  to  have  at  least 
ten  observations  by  the  end  of  the  exercise. 

b.  Some  standards  refer  to  unit  actions  which  may  be  required 
of  more  than  one  unspecified  soldier  or  element.  This  situation  is  similar  to 
the  one  discussed  in  paragraph  “a."  Here,  however,  the  evaluator's  problem  is 
how  to  evaluate  the  performance  if  some  of  the  relevant  soldiers  or  elements 
perform  the  required  actions  while  others  do  not. 

EXAMPLES:  "THE  PLATOON  OCCUPIES  CONCEALED,  HULL  DOWN  POSITIONS..." 

(ARTEP  71-2,  3-IV-1-10) 

j  “THE  PLATOON:... PERFORMS  OPERATOR  MAINTENANCE  ON 

j  VEHICLES,  WEAPONS  AND  OTHER  EQUIPMENT."  (ARTEP  71-2, 

I 

3-IV-8-U) 

SUGGESTIONS:  For  standards  that  refer  to  actions  required  of  all 

or  most  of  a  unit,  use  the  70%  rule.  That  is,  if  at  least  70  percent  of  the 
elements  required  to  perform  an  action  actually  do  so,  the  evaluator  should 
indicate  that  the  standard  has  been  met.  The  evaluator  should  also  make  notes 
on  elements  who  do  not  perform  the  action  required. 

Some  standards  appear  similar  in  format  to  .he  examples  shown 
above,  but  are  actually  standards  for  leader  actions  or  for  actions  of  one  or 
two  elements  within  a  unit.  If  a  leader  action  is  satisfactorily  completed, 
whether  by  the  leader  himself  or  by  a  unit  member,  the  evaluator  should 
indicate  that  the  standard  has  been  met.  Similarly,  if  an  action  is  required 
of  only  one  or  two  elements  within  a  unit,  the  standard  should  be  treated  in 
the  same  way.  That  is,  if  the  required  action  is  satisfactorily  completed. 
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the  evaluator  should  indicate  that  the  standard  has  been  met.  Here  again,  the 
evaluator  should  make  notes  on  unusual  or  extenuating  circumstances. 

In  cases  where  failure  to  perform  to  standard  by  elements  within  a 
unit  is  exceptionally  severe,  the  situation  should  be  brought  to  the  attention 
of  the  senior  evaluator  as  soon  as  the  exercise  is  completed. 

c .  Some  standards  refer  to  action  sequences  which  begin  with 
cues.  The  cues  are  often  difficult  for  the  evaluator  to  detect.  These  occur 
especially  under  tasks  that  are  related  to  engagements. 

i  EXAMPLE:  “OVERMATCHING  ELEMENTS  INITIATE  ENGAGEMENT  WITHIN  FIVE 

SECONDS  AFTER  TARGET  APPEARANCE.**  (ARTEP  71-2,  3-IV-7- 

j  . 

;  3)  This  standard  requires  the  evaluator  to  detect  the 

!  target  the  second  its  inLervisibie . 

1  “(PLATOON)  REQUESTS  IMMEDIATE  SUPPRESSION  OF  OPFOR 

I 

POSITION."  (ARTEP  71-2,  3-IV-2-6)  This  standard 
requires  the  evaluator  to  detect  the  OPFOR  cue 
immediately,  to  be  listening  to  the  tactical  net  at 

the  moment  the  request  occurs,  and  to  know  the  precise 

i 

location  of  the  OPFOR  position. 

SUGGESTIONS:  If  the  evaluator  misses  the  cue  entirely,  he  should 

indicate  “not  observed. “  If  the  evaluator  sees  the  cue  late  and  the  standard 
refers  to  a  time  interval,  the  evaluator  should  begin  to  time  the  event  when 
he  detects  the  cue.  He  should  not  try  to  guess  when  the  cue  first  appeared 
and  make  judgement  on  that  basis.  Where  terms  are  vague  (e.g.,  "immediate"), 
the  senior  evaluator  should  supply  the  definition. 

d.  Some  standards  require  the  evaluator  to  know  what  is 
happening  on  both  sides  of  the  exercise.  These  are  usually  encountered  in 
engagement-related  tasks. 
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EXAMPLE:  "THE  ASSUALT  SECTION  COMMENCES  THE  ASSULT  WHEN  SUPPORTING 


|  AND  DIRECT  FIRES  HAVE  SUPPRESSED  THE  OPFOR  POSITON." 

(ARTEP  71-2,  3-IV-2-8)  This  standard  requires  the 
evaluator  to  know  when  the  OPFOR  is  suppressed. 

SUGGESTIONS:  Keep  in  close  contact  with  OPFOR  evaluator  and  the 

Fire  Marker  team,  if  one  is  used.  When  evaluating  the  effects  of  unit  actions 
while  using  MILES,  ask  the  evaluator  with  the  OPFOR  or  Fire  Marker  team  leader 
whether  the  OPFOR  is  receiving  fires  and  what  the  effects  are.  For  example, 
if  suppressive  direct  fire  is  under  way,  the  OPFOR  should  be  receiving  MILES 
“near-miss"  indications  whenever  exposed.  Also,  make  sure  that  the  unit's 
weapons  are  pointed  at  the  OPFOR,  because  another  unit  may  be  responsible  for 
the  suppressive  or  other  effects  reported  by  those  near  the  OPFOR. 

e.  Some  standards  may  not  be  appropriate  under  some 
conditions.  The  most  frequently  occuring  reason  is  that  the  terrain  may  not 
permit  the  unit  to  execute  certain  actions.  Other  factors  (e.g.,  weather)  may 
also  make  some  standards  inappropriate. 

EXAMPLES:  "(THE  PLATOON)  REQUESTS  SUPPRESSIVE  FIRE  AND  SMOKE  ON 

THE  ATTACKING  OPFOR."  (ARTEP  71-2,  3-IV-3-9)  High  wind 
j  or  wind  blowing  in  the  wrong  direction  may  make  the  use 

i  of  smoke  inappropriate. 

i 

|  "THE  LEAD  SQUAD  MOVES  CONTINUOUSLY  ON  A  COVERED, 

“ ^ ™ * •mmmmm — -w— m — ■— — — — — — 

CONCEALED  AXIS  100  TO  400  METERS  FORWARD  OF  THE  LEAD 
VEHICLE  OF  THE  TRAIL  ELEMENT."  (ARTEP  71-2,  3-IV-6-8) 
Terrain  usually  does  not  permit  this  standard  to  be 
fully  met.  It  is  rare  that  a  fully  covered  movement 
route  is  available. 
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SUGGESTION:  If  a  standard  cannot  be  met  because  of  conditions  over 


which  the  evaluated  unit  has  little  or  no  control,  do  not  evaluate  the  unit  on 
that  standard.  Indicate  "not  evaluated"  on  evaluation  form  and  make  a  note  of 
the  reason.  If  a  unit's  action  seems  to  be  in  violation  of  the  principle 
underlying  a  standard,  make  a  special  note  and  review  the  circumstances  with 
the  senior  evaluator  after  the  exercise.  For  example,  suppose  the  terrain  did 
not  permit  a  unit  to  move  on  a  continuously  covered,  concealed  route.  But 
suppose  also  that  the  unit  did  not  use  t'ue  cover  or  concealment  which  was 
available.  Even  though  the  evaluator  might  indicate  “not  observed,"  he  should 
also  bring  the  performance  to  the  attention  of  the  senior  evaluator.  While 
uniform  application  of  performance  standards  is  important,  the  primary  goal  is 
to  determine  what  Luc  unit  r.eed«  mnst  to  focus  on  in  later  training  exercises. 

f .  Some  standards  refer  to  actions  which  are  often  not 
observable.  These  are  most  commonly  found  under  tasks  which  deal  with 
planning,  and  command  and  control,  and  often  refer  to  the  leaders  thought 

processes.  _ 

/ 

EXAMPLES:  "UNIT  PERSONNEL  PERFORM  THE  BACKWARD-PLANNING 
SEQUENCE..."  (ARTEP  71-2,  3-1-1). 

"COMMANDERS  AND  STAFF  ANALYZE  UNIT  ACTIVITIES..." 

(ARTEP  71-2,  3-1-2). 

"THE  MANEUVER  ELEMENT  OCCUPIES  THE  SUBSEQUENT  OVERWATCH 
POSITION  AND  CONDUCTS  A  RAPID  VISUAL  SEARCH  OF  THE 
POSITION  AND  ADJACENT  TERRAIN."  (ARTEP  71-2,3-V-3-10) 


Because  much  of  the  activity  referred  to  by  such  standards  is  not 
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necessarily  visible,  such  standards  Lend  to  be  difficult  to  apply.  The 
evaluator  should  look  for  interactions  among  the  unit  members,  and  concrete 
products  which  indicate  the  required  activity  is  taking  place.  For  example, 
if  a  standard  requires  analysis  of  unit  activities,  indicators  might  include 
conversations  among  key  leaders  or  written  notes.  Unless  there  is  some 
concrete,  observable  behavior  which  clearly  indicates  that  the  action  referred 
to  by  the  standard  has  been  omitted  or  completed,  the  evaluator  should 
indicate  "not  observed"  on  the  evaluation  form.  The  evaluator  should  not  Lry 
to  infer  whether  or  not  a  standard  is  met  based  on  other  subsequent  actions  or 
outcomes. 

g.  Some  standards  do  not  explicitly  indicate  what  actions  the 
evaluator  is  to  look  for. 

S  EXAMPLES:  "TARGETS  ARE  ENGAGED  WITH  THE  MINIMUM  WEAPONS  NECESSARY 

!  TO  INSURE  DESTRUCTION."  (ARTEP  71-2,  3-V-3-7). 

I 

I  "(THE  PLATOON)  SETS  UP  LOCAL  SECURITY."  (ARTEP  71-2,  3- 

1 

|  IV-13-1 ) 

i 

"(THE  SQUAD)  ESTABLISHES  AN  OBSERVATION  POST  (OP)  ON 
ORDER,  TO  PROVIDE  EARLY  WARNING  OF  OPFOR  ACTIVITY  IN  THE 

V  .vREA."  (ARTEP  71-2,  3-III-3-2) 

SUGGESTION:  These  types  of  standards  depend  heavily  on  Lhe 

evaluator's  tactical  knowledge  and  should  be  thoroughly  covered  during  the 
evaluator  training.  The  evaluation  team  leader  should  give  the  evaluators 
several  specific  examples  of  both  adequate  and  inadequate  unit  actions. 
Examples  should  be  given  in  terms  of  the  tactical  scenario  to  be  used  in  the 
exercise.  If  evaluators  have  difficulty  in  applying  standards  during  an 
exercise,  the  problems  should  be  discussed  with  the  senior  evaluator  after  the 
exercise. 
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Stage 


Step 


STEP  2.  Terminate  Exercise 


1.  It  is  sometimes  appropriate  to  terminate  an  exercise  before  it 
is  completed.  The  decision  for  early  termination  depends  on  the  evaluation 
goals  and  on  the  attrition  in  the  unit  and  OPFOR. 

a.  In  internal  evaluations  the  exercise  can  be  terminated  when 
either  of  the  forces  is  not  making  any  significant  progress  or  when  either  of 
the  forces  has  sustained  enough  casualties  to  make  further  progress 
unlikely.  It  is  not  effective  training  to  have  units  fight  to  the  last  man. 
Generally,  if  a  unit  has  in  excess  of  30  percent  casualties,  it  will  have  an 
extremely  difficult  time  completing  its  mission.  More  importantly,  little 
will  be  gained  by  prolonging  the  evaluation  when  a  high  proportion  of  the 
players  are  casualties:  there  will  be  a  few  soldiers  left  in  the  exercise  who 
can  be  evaluated. 

b.  In  external  evaluations  the  above  considerations  apply. 
However,  in  some  cases  there  may  be  a  requirement  to  complete  the 
evaluation.  If  the  unit  has  sustained  a  high  percentage  of  casualties  early 
in  the  exercise,  it  may  be  advisable  to  (a)  halt  the  exercise,  (b)  withdraw 
the  OPFOR  and  reset  the  MILES  devices,  (c)  allow  the  unit  to  bring  its 
casualties  forward  and  reorganize,  and  then  (d)  to  restart  the  exercise.  A 
typical  case  which  calls  for  this  procedure  is  when  the  unit  is  required  to 
conduct  an  attack  and  then  to  consolidate  and  reorganize.  If  the  unit 
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sustains  heavy  casualties  before  seizing  their  objective,  there  is  likely  to 
be  an  inadequate  basis  for  a  meaningful  evaluation  of  the  unit's  capability  to 
carry  out  consolidation  and  reorganization  tasks. 

2.  When  the  exercise  is  terminated,  evaluators  should  collect  data 
on  the  exercise  outcome  which  may  have  a  bearing  on  later  interpretations. 
These  data  car.  include: 

a.  Number  and  types  of  casualties  on  each  side. 

b.  Rounds  expended  (from  MILES  equipped  tanks,  TOWs  and 
DRAGONs). 

c.  Type  of  weapon  inflicting  kills  (MILES  equipped  tanks,  APCs 

and  TOWs). 

d.  Final  disposition  of  forces. 

e.  Unusual  conditions  or  events  (e.g.,  weather;  elements 
firing  after  exercise  was  terminated). 

f.  Tactical  vehicles  not  operational  (mechanical). 

3.  Most  of  the  evaluation  .. _ems  should  have  been  completed  during 
the  exercise.  However,  some  may  have  to  wait  until  the  exercise  is 
terminated.  There  are  two  types  of  such  items:  (a)  those  which  refer  to 
actions  which  should  occur  repetitively  or  continuously  throughout  the 
exercise,  and  (b)  those  about  which  the  evaluator  was  uncertain  of  how  to 
score  the  unit.  Items  about  which  the  evaluator  is  uncertain  should  be 
discussed  with  the  senior  evaluator  before  scoring  them.  If  an  After  Action 
Review  is  to  be  held,  these  items  can  be  discussed  in  the  controller  debrief 
(STAGE  IV,  STEP  1.) 

STAGE  IV.  "After  Exercise"  includes  two  steps: 

STEP  1.  Prepare  the  After  Action  Review 
STEP  2.  Conduct  After  Action  Review 
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Stage  Step 


site  needs  tc  be  selected  for  the  AAR.  If  possible,  the  AAR  should  be  held 
where  the  majority  of  action  occurred,  where  the  most  critical  events  took 
place,  or  where  this  terrain  can  be  observed.  Most  often  the  GrrOR  objective 
or  the  unit  objective  will  be  suitable  for  assembling  the  players  and 
conducting  the  AAR. 

2.  Debrief  Controller.  After  the  exercise,  and  the  necessary 

troop  leading  procedures,  the  evaluator/AAR  leader  should  review  his  knowledge 
about  the  critical  events  and  determine  the  nature  of  major  information 
gaps.  The  evaluator/AAR  leader  must  have  a  complete  understanding  of  what 
happened  in  the  exercise,  from  the  unit  entering  its  initial  positions  through 
termination  of  the  exercise.  Obtain  a  detailed  description  of  the  exercise's 
major  tactical  events  in  the  order  in  which  they  occurred.  Descriptions 

should  emerge  from  the  debriefing  of  the  subordinate  unit  evaluators  and  the 

OPFOR  evaluator(s).  All  evaluators  should  be  encouraged  to  contribute  their 
observations  regarding  the  elements  for  which  they  were  responsible.  The 

following  examples  of  topics  about  which  the  evaluator  should  have  relatively 
detailed  information  may  be  helpful. 

a.  Important  aspects  of  mission  planning  and  preparation. 

i 

b.  Disposition  of  forces. 

c.  FRAGOs  involving  major  changes  on  plans. 
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d.  Deviations  from  planned  routes  and/or  actions. 

e.  Major  engagements  and  their  results. 

f.  Coordination  and  communications. 

3.  After  the  evaluator  has  a  good  understanding  of  what  happened 

during  the  exercise,  he  should  review  the  critical  events  and  rank  them  in 

terms  of  their  relevance  to  the  exercise  training  objectives  and  Lheir 

contribution  to  the  exercise  outcome.  He  should  then  select  as  many  critical 
events  as  can  be  covered  in  detail  during  the  Lime  allowed  for  Lhe  AAR  and 
place  them  in  chronological  order.  Writing  key  words  on  the  T&EO  or  ETQ  form 
may  help  the  evaluator  to  guide  the  AAR  and  keep  the  discussion  focused. 

4.  Select  teaching  points  to  be  made  in  AAR.  Teaching  points 

should  focus  on  critical  events  that  occurred  during  the  exercise.  A  critical 
tactical  event  is  often  related  to  a  major  loss  or  gain  that  impairs  or 
enhances  a  unit's  ability  to  perform.  In  MILES  exercises,  critical  events  are 
usually  associated,  one  way  or  anoLher,  with  casualties  inflicted  or 

sustained.  After  the  AAR  leader  has  filled  in  any  gaps  in  his  knowledge  of 
the  exercise,  he  matches  teaching  points  to  be  made  with  Lhe  sequence  of 
critical  tactical  events.  Tactical  events  can  provide  Leaching  points  "of 
opportunity"  and  these  may  be  included  if  important.  However,  discussions 
unrelated  to  important  teaching  points  should  be  avoided.  At  this  point,  the 
AAR  leader  should  have  a  list  of  key  words  as  reminders  of  teaching  points 

critical  tactical  events.  This  includes  the  following  for  each  event. 

-  What  Happened  -  description  of  the  critical  event. 

-  How  it  Happened  -  key  facts  surrounding  the  critical  event. 

-  Why  it  Happened  -  inferences  about  probable  causes. 


-  Alternative  Courses  of  Action  -  how  could  the  unit  have  done 


better. 

5.  Evaluation  data  forms  should  be  collected  from  subordinate  unit 
evaluators,  and  briefly  reviewed  for  completeness.  If  evaluators  have  kept 
their  notes  on  something  other  than  the  evaluation  forms,  they  should  also  be 
collected  and  attached  to  the  appropriate  form* 
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STEP  2.  Conduct  the  After  Action  Review 


1.  Organize  the  participants.  When  the  evaluator/AAR  leader 
assembles  the  participants  for  the  AAR  he  should  organize  them  according  to 
their  organization  in  the  exercise.  Each  subordinate  element  controller 
should  be  with  the  unit  for  which  he  was  responsible.  All  players  should  be 
present  for  squad  and  platoon  AARs.  In  a  company- level  AAR  not  all  players 
should  be  present.  The  company  AAR  is  directed  toward  the  leaders.  In  an 
armor  company,  players  from  the  tank  commanders  up  should  be  present.  For  an 
infantry  company,  players  from  the  squad  leaders  up  should  be  present.  Other 
key  players  should  be  present  as  needed.  For  example,  the  FIST  leader  should 
be  present  if  friendly  indirect  fire  was  included  in  the  exercise.  The 
remaining  troops  should  be  released  back  to  their  vehicles  for  maintenance, 
preparation  for  the  next  exercise,  etc.  Figure  2.  shows  the  physical  layout 
for  an  infantry  company  AAR. 
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2.  Begin  the  AAR  by  asking  the  unit  leader  to  make  a  brief 
statement  of  the  training  objective.  These  should  be  described  as 
specifically  as  possible.  The  AAR  leader  should  also  state  any  additional 
teaching  points  that  he  intends  to  cover  during  the  AAR.  The  number  of  these 
should  be  limited  to  three  or  four  key  ones  to  keep  the  AAR  focused  and 
prevent  it  from  becoming  excessively  long. 

3.  Next  guide  a  discussion  of  the  major  tactical  events,  in  their 
order  of  occurrence.  Diagrams  should  be  employed  to  help  players  visualize 
the  exercise  development.  Start  by  sketching  the  main  terrain  features  and, 
as  the  AAR  proceeds,  have  the  participants  draw  in  routes  of  advance, 
objectives,  locations  of  engagements,  etc.  A  general  scenario  for  a  company- 
level  AAR  is  shown  in  Table  3.  Squad  and  platoon  AARs  follow  the  same  general 
sequence,  though  the  focus  will  be  more  on  execution  than  on  command  and 
control  issues  at  the  lower  echelons. 

4.  Each  major  event  should  be  discussed  in  detail  to  make  teaching 
points  about  the  unit's  performance  during  the  event.  The  AAR  leader  does  the 
following  in  an  effective  AAR: 

a.  Avoids  giving  a  critique  or  a  lecture. 

b.  Guides  the  discussion  by  asking  leading  questions. 

c.  Has  players  describe  what  happened  in  their  own  terms. 

d.  Has  players  discuss  not  only  what  happened  but  how  it 
happened,  why  it  happened,  and  how  it  could  have  been  done  better. 

e.  Focuses  the  discussion  so  that  important  tactical  lessons 
are  made  explicit. 

f.  Relates  tactical  events  to  subsequent  results. 

g.  Avoids  detailed  examination  of  events  not  directly  related 
to  major  training  objectives. 
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2d  Platoon 


| Cont .  SLs  PL 


(SL  =  Squad  Leader,  PL  =  Platoon  Leader,  Cont.  =  Controller) 


i 


Figure  2,  Arrangement  for  infantry  company  after  action  review  (AAR) 


Table  3 


General  Scenario  for  A  Company  AAR 


Event 

State  Training  Objectives 
or  Teaching  Points 

OPFOR  Plan 

Company's  Plan 

Events  before  Detection/ 
Contact 

First  Delection/Contact 
Frag  Orders 

Events  During  Engagement 

Final  Results 

Summary 


Responsibility 

Company  Commander  and  AAR  Leader 

OPFOR  Leader 

Company  Commander 

Company  Commander/Platoon  Leaders 

Participants 
Company  Commander 
All  Participants 
All  Participants 
AAR  Leader 


h.  Encourages  the  participants  to  use  diagrams  to  illustrate 
teaching  points  and  to  show  routes,  phase  lines,  objectives,  etc. 

i.  Does  not  allow  players  to  offer  self-serving  excuses  for 
inappropriate  tactical  actions. 

5.  The  AAR  leader  briefly  summarizes  teaching  points  in  terms  of 
the  training  objectives  covered  in  the  AAR.  After  the  summary,  the  AAR  leader 
can  have  a  private  conversation  with  the  unit  leader  regarding  his  strengths 
and  weaknesses  and  what  he  can  do  to  further  improve  his  performance,  and  that 
of  his  unit.  Whenever  possible,  an  opportunity  should  be  provided  for  the 
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unit  leader  to  discuss  the  points  raised  in  the  AAR,  as  well  as  his  own 
observations,  with  the  members  of  his  unit. 

6.  Characteristics  of  a  good  AAR: 

a.  Order  and  discipline  are  maintained. 

b.  Training  objectives  are  reviewed. 

c.  The  AAR  leader  guides  unit's  discussion  to  the  important 
events,  reasons  why  these  occurred,  and  how  the  unit  could  have  done  better. 
Avoids  detailed  examination  of  events  not  directly  related  to  training 
objectives. 

d.  The  AAR  leader  traces  chains  of  events  so  that  the  results 
of  mistakes  are  understood  by  participants  (one  mistake  is  often  a  partial 
cause  of  another). 


e.  Attention  of  the  participants  is  held  and  they  are 
involved  in  the  discussion. 

f.  The  summary  and  new  training  objectives  are  clear  and 


concise. 


g.  Sketches  or  diagrams  are  used  to  reinforce  points  made  in 


the  AAR. 
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STAGE  V. 


"After  Return  to  Garrison"  consists  of  three  steps: 


STEP  1.  Summarize  Results 

STEP  2.  Review  Final  Results 

STEP  3.  Prepare  Report  or  Briefing 


1  STEP  1.  Summarize  Results 


1.  Organize  the  results  by  units  one  echelon  below  the  unit 

evaluated. 

2.  Review  the  results  for  completeness. 

3.  Identify  results  which  seem  unusual,  or  which  suggest  some 
evaluation  problem.  For  example,  if  a  platoon  was  evaluated  and  two  squads 
did  very  well,  but  one  squad  had  serious  problems,  carefully  review  the  third 
squad's  results,  and  determine  the  evaluator's  level  of  experience  or  other 
possible  extenuating  circumstances. 

4.  Summarize  the  results  for  sub-echelon  by  evaluation  standard. 
Calculate  the  number  or  percentage  of  positive,  negative  and  "not  evaluated" 
responses  for  each  standard,  for  each  task,  and  for  each  major  subordinate 
eche Ion. 

5.  Identify  tasks  or  groups  of  standards  which  seemed  to  give  the 
unit  the  most  difficulty.  Also  note  tasks  on  which  the  unit  did  particularly 
well. 
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6.  Review  evaluators'  notes.  Review  notes  concerning  the  unit's 
problem  areas  and  try  to  determine  the  reasons  for  their  problems.  If  a 
problem  seems  to  be  caused  by  a  scenario,  terrain,  evaluator,  or  equipment 
difficulties,  make  comments  to  that  effect  in  your  summary. 

7.  Make  a  list  of  the  unit's  strong  points  and  one  of  their  weak 

points. 


Stage  Step 


1.  Review  the  evaluation  forms  and  notes  with  the  evaluation  team, 
standard  by  standard.  Separate  unit  performance  problems  from  those  caused  by 
other  factors  (scenario,  exercise  control,  terrain,  evaluators,  training 
devices  or  unit  equipment). 

2.  Review  your  lists  of  strong  and  weak  points  with  the  team,  and 
make  changes  as  appropriate. 


Stage  Step 


1.  Results  should  be  interpreted  in  a  way  which  helps  unit  leaders 
to  conduct  further  training.  Naturally,  remedial  training  should  be  centered 
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on  the  unit's  weak  points.  Therefore,  the  unit's  problems  should  be  explained 
most  thoroughly. 

2.  Information  from  evaluations  can  be  broken  into  three 

categories: 

a.  Information  from  unit  performance  on  T&EO  tasks  (including 
notes  and  comments). 

b.  Mission  Accomplishment. 

c.  Casualties  inflicted  and  sustained. 

3.  Normally,  these  three  types  of  information  will  be  consistent 
with  one  another.  Only  rarely  should  one  find  any  major  differences  which 
have  not  been  resolved  in  an  earlier  step.  When  inconsistencies  do  occur, 
they  are  most  often  caused  by  problems  outside  the  un  being  evaluated.  Some 
common  sources  of  inconsistent  results  are  discussed  below. 

a.  The  tactical  scenario  can  be  too  difficult.  This  usually 
results  in  a  very  poor  casualty  exchange  ratio  and  mission  failure.  If  the 
unit  is  well  trained,  the  evaluator  may  notice  that  even  though  the  unit  had  a 
poor  casualty  exchange  ratio  and  failed  to  accomplish  its  mission,  the  unit 
satisfactorily  completed  most  of  the  T&EO  tasks.  Other  factors  which  may 
suggest  an  overly  difficult  tactical  scenario  include  extremely  long  or  short 
exercises.  In  the  former  case,  a  unit  may  become  pinned  down  with  little  or 
no  opportunity  to  maneuver  without  sustaining  heavy  casualties,  thus 
prolonging  the  exercise.  Very  short  exercises  can  occur  when,  for  example, 
the  OPFOR  is  able  to  bring  heavy,  sustained  fire  against  the  unit  early  in  the 
exercise,  the  terrain  does  not  provide  cover  and  the  scenario  does  not  provide 
the  unit  with  access  to  supporting  fire. 

b.  The  tactical  scenario  can  be  too  easy.  This  usually 
results  in  an  exceptionally  good  casualty-exchange  ratio,  mission 
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accomplishment  and  very  short  exercises.  If  the  unit  is  less  than  "well 
trained"  these  results  will  be  inconsistent  with  performance  on  T&EO  tasks. 

c.  Factors  which  are  usually  responsible  for  the  scenario 
being  too  difficult  or  easy  are  the  terrain,  the  force  ratios,  the  distances 
between  the  unit  and  OPFOR,  and  inclusion  of  too  many  tasks  or  events.  Also, 
repeated  OPFOR  experience  on  the  same  terrain  can  cause  unfavorable  casualty 
exchange  raLios  and/or  mission  failure  and  appears  as  an  inconsistency  when 
few  performance  problems  are  found  on  T&EO  tasks. 

d.  Problems  with  MILES  equipment  usually  results  in  low 
numbers  of  casualties  on  both  sides  of  the  engagement.  Another  typical 
symptom  of  MILES  equipment  problems  is  the  unit  and  OPFOR  getting  "tangled-up" 
at  the  end  of  an  exercise  without  substantial  casualties.  Performance  data 
from  T&EO  tasks  may  not  bear  any  consistent  relation  to  casualty  and  mission 
accomplishment  data  in  such  cases. 

e.  Problems  with  unit  equipment  (especially  tactical  vehicles) 
often  results  in  a  poor  casualty  exchange  ratio  and/or  mission  failure.  Here 
again,  T&EO  task  data  may  not  bear  any  consistent  relation  to  casualties  or 
mission  accomplishment. 

f.  Leader  performance  problems,  especially  inappropriate 
decisions  can  often  result  in  a  poor  casualty  exchange  ratio  apd/or  mission 
failure,  sometimes  with  few  indications  of  performance  problems  on  T&EOs. 
These  indications,  however,  are  usually  concentrated  in  the  standards  related 
to  command  and  control. 

4,  Prepare  Report  or  Briefing.  Most  units  will  have  a  format  SOP 
for  preparation  of  After  Action  Reports.  However,  the  report  or  briefing 
contents  should  cover  the  following: 

a.  Outline  of  Scenario. 
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b.  Outcome  of  Exercise. 

c.  Major  areas  in  need  of  further  training. 

d.  Recommended  training  strategy. 

e.  Problems  encountered  during  evaluation  (lessons  learned). 
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APPENDIX  A 


AFTER  ACTION  REVIEW  -  BACKGROUND  AND  TECHNIQUES 


APPENDIX  A 


AFTER  ACTION  REVIEW  -  BACKGROUND  AND  TECHNIQUES 

The  procedures  for  preparing  and  conducting  After  Action  Reviews  (AAR) 
were  presented  in  STAGE  IV  of  the  preceding  section.  This  section  provides  a 
rationale  for  AARs  and  some  general  style  and  technique  considerations.  This 
material  can  be  used  by  evaluation  team  members  who  desire  either  a  broader 
perspective  on  performance  feedback  or  a  rationale  for  the  procedures  given  in 
STAGE  IV. 

BACKGROUND  AND  RATIONALE 

After  a  tactical  training  exercise,  feedback  should  be  provided  to  units 
in  order  to  increase  and  reinforce  learning.  In  the  past,  feedback  has  been 
given  during  a  critique  in  which  the  senior  evaluator  presents  his  analysis  of 
the  unit's  performance  and  indicates  what  the  unit  did  well  and  whaL  they  did 
poorly.  In  a  good  critique,  the  evaluator  also  indicates  training  strategies 
for  correcting  the  unit's  major  problems.  Before  the  development  of  tactical 
engagement  simulation  training  methods  (e.g.,  MILES),  the  critique  was  t'.e 
principal  method  for  informing  units  about  their  levels  of  proficiency.  In 
times  past,  the  critique  was  an  adequate  solution  to  the  feedback  problem 
because  the  scarcity  of  objective  performance  data  made  extensive 
interpretation  of  tactical  events  necessary. 
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Tactical  engagement  simulation  training  systems  began  to  be  developed 
during  the  early  1970s.  These  methods,  characterized  by  reasonably  accurate 
weapon  effects  simulations,  provided  the  opportunity  to  replace  the  critique 
with  a  more  effective  teaching  technique.  In  order  to  distinguish  it  from  the 
lecture-format  critique,  the  new  feedback  method  was  called  the  After  Action 
Review  (AAR).  The  following  comparisons  explain  the  nature  of  the  AAR  by 
contrasting  it  with  the  familiar  critique. 

The  AAR  Increase  Soldier  Participation.  In  a  critique,  commanders  and 
soldiers  are  basically  an  audience;  in  an  AAR,  they  are  participants.  This 
difference  dramatically  increases  Leaching  effectiveness  for  three  reasons. 
First,  active  participation  in  a  learning  activity  (as  opposed  to  passive 
observation)  greatly  increases  the  amount  of  information  learned  and 
retained.  Second,  In  a  discussion,  topics  are  often  approached  from  several 
points-of-view,  thus  increasing  the  chance  that  participants  will  gain  greater 
insight  into  the  topic  at  hand.  In  contrast,  only  one  point-of-view  is 
presented  in  a  critique — that  of  the  lecturer — and  the  chances  that  a  large 
proportion  of  the  audience  will  benefit  are  substantially  less.  Finally, 
direct  participation  increases  motivation  by  providing  a  sense  of  involvement 
in  the  learning  process.  Such  involvement  frequently  reduces  a  soldier's 
resistance  to  acknowledging  his  own  mistakes,  thereby  further  increasing 
learning  and  retention  of  tactical  skills. 

The  AAR  is  Broad  in  Scope.  In  a  critique,  the  leader  is  limited  by  the  type 
and  amount  of  information  he  and  perhaps  a  few  others  have  gathered.  In 
contrast,  because  all  key  players  participate  in  an  AAR,  each  is  a  source  of 
information.  Thus,  the  AAR  inherently  provides  a  much  richer  "data  base"  from 
which  teaching  points  can  be  drawn.  This  is  especially  critical  at  command 
levels  because  much  important  information  is  essentially  private.  For 
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example,  the  commander's  assessment  of  the  situation  and  the  reasons  for  his 
tactical  decisions  are  available  only  to  him.  In  a  critique,  this  kind  of 
information  is  most  often  not  taken  into  account.  In  the  AAR,  however,  such 
information  is  an  important  part  of  the  discussion  and  forms  the  context  for 
discussing  alternative  courses  of  action. 

The  AAR  Structure  is  Easy  to  Follow.  The  AAR  is  structured  around  sequential 
exercise  events.  This  helps:  (a)  examination  of  chains  of  events,  (b) 
determination  of  how  and  why  specific  actions  were  undertaken,  (c)  active 
discussion  of  alternatives,  and  (d)  examination  of  how  certain  events 
determined  or  influenced  subsequent  outcomes.  The  exercise  event-oriented  AAR 
structure  is  based  on  the  recognition  that  unit  leaders  and  soldiers  need  to 
learn  that:  (a)  no  matter  what  the  situation  may  be,  alternative  courses  of 
action  exist,  and  ( b)  leaders  and  soldiers  should  select  from  among  these 
alternatives  after  evaluating  what  the  probable  consequences  of  each  would 
be.  This  is  distinctly  different  from  a  critique  in  which  "failures'*  are 
often  pointed  out,  but  actions  that  influenced  or  determined  failure  are 
rarely  explored  in  detail.  In  a  critique  the  actions  needed  to  avoid 
"failure"  are  frequently  not  clear  to  unit  leaders  or  soldiers. 

Because  the  specific  topics  discussed  within  the  context  of  a  particular 
scenario  are  directly  determined  by  a  unit's  tactical  behavior,  the  AAR  is  a 
highly  flexible  teaching  vehicle.  A  wide  variety  of  tactical  actions  and 
training  objectives  can  be  explored  and  evaluated  depending  upon  the  unit's 
particular  training  needs.  The  AAR  struture  provides  a  sequential,  easy  to 
follow  framework  and  helps  soldiers  to  explore  important  training  issues. 

The  AAR  Increases  the  Accuracy  of  Interpretation.  Points  made  during  a 
critique  will  often  be  based  solely  on  the  analysis  of  the  leader  conducting 
it.  His  analysis  will  often  be  based  on  limited  information  on  the  local 


tactical  situation,  guesses  regarding  the  unit's  intention,  and  limited 
knowledge  regarding  information  available  to  the  element  or  leader  at  the  time 
of  the  action  or  decision.  In  an  AAR,  these  limitations  are  overcome  through 
direct  player  participation.  Important  players  are  asked  about  what  they  knew 
at  specific  points  in  the  exercise,  their  situation  assessments,  why  certain 
tactical  decisions  were  made,  and  so  on.  These  kinds  of  questions  and  answers 
lead  to  more  accurate  interpretation  of  exercise  events,  better  training 
diagnosis  and  more  fruitful  discussions  of  alternative  courses  of  action.  (A 
detailed  example  is  given  in  Table  A-l  p.  47). 

The  AAR  Avoids  Negativism.  In  contrast  to  the  lecture  format  of  a  critique, 
the  AAR  leader  guides  the  discussion  by  asking  leading  questions.  Except  for 
making  periodic  summaries,  the  AAR  leader  rarely  makes  a  declarative 
statement.  Key  information  is  brought  out  by  questioning  as  many  of  the 
relevant  soldiers  and  leaders  (on  both  sides)  as  needed  to  make  a  point.  Once 
a  critical  action  (or  decision)  is  identified,  furLher  questions  explore  why 
the  action  was  taken,  its  consequences,  and  what  alternative  existed.  This 
questioning  technique  involves  participants  in  the  examination  of  the  problem 
and  avoids  difficulties  of  resentment  and  resistance  usually  generated  by 
direct  criticism.  By  asking  questions  rather  than  lecturing,  the  AAR  leader 
sets  the  tone  of  the  AAR  as  a  group  problem  solving  session  among  fellow 
professional  soldiers.  Even  though  the  AAR  leader  knows  the  unit's  mistakes, 
he  guides  the  participants  to  identify  errors  themselves  and  to  seek 
solutions.  Because  the  information  comes  from  within  the  group,  hostility  and 
defensiveness  often  directed  towards  the  critique  leader  are  minimized.  In 
the  critique,  the  central  theme  is  "What  you  did  wrong.”  In  the  AAR,  the  key 

i 

thrust  is  ”How  can  we  do  better?”  The  latter  orientation  is  by  far  the  most 
preferable.  By  involving  appropriate  commanders,  leaders,  and  troops  in  a 
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professional  discussion  of  "How  can  we  do  better?",  the  cohesiveness  of  the 
unit  and  the  chain  of  command  are  simultaneously  reinforced. 

AAR  TECHNIQUE  AND  STYLE 

Tactical  Engagement  Simulation  training  often  fosters  high  degree  of 
enthusiasm  among  the  troops.  In  most  ways  the  enthusiasm  is  good,  but  it  can 
make  it  difficult  to  lead  a  good  AAR.  A  reasonable  amount  of  order  and 
discipline  must  be  maintained.  The  following  suggestions  may  be  helpful. 

1.  Encourage  the  troops  to  talk  among  themselves  during  Lhe 
Controller/Evaluator  Debriefing.  It  may  help  to  eliminate  some  of 
the  chatter  later. 

2.  Inform  the  troops  that  the  basic  AAR  rules  are  that: 

a.  Only  one  person  talks  at  a  Lime; 

b.  Only  the  individual  designated  by  Lhe  AAR  leader  talks; 

c.  Soldiers  who  want  to  make  comments  should  raise  their  hands  and 
wait  to  be  called  upon; 

d.  Keep  on  track.  Comments  will  only  be  accep.ed  on  the  topic 
being  discussed. 

The  point  was  made  earlier  that  one  avoids  lecturing  in  an  AAR  and 
instead  asks  leading  questions.  The  questioning  technique  avoids  the  problems 
of  resentment  and  resistance,  fosters  positive  motivation,  and  allows  in-depth 
exploration  of  training  objective-related  issues.  The  AAR  leader's  questions 
are  most  often  those  to  which  he  already  knows  the  answer.  Asking  questions 
is  simply  a  device  for  drawing  those  answers  from  the  group.  That  way, 
information  and  comments  come  directly  from  participants  rather  than  being 
criticism  from  the  AAR  leader. 

In  a  sequence  of  questions  on  a  given  point,  the  first  few  questions  are 
intended  to  help  the  group  identify  an  important  event  or  problem.  The  next 
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questions  serve  to  elaborate  and  clarify  the  circumstances  and  causes  of  the 
event.  Final  questions  help  the  group  explore  alternative  courses  of  the 
event.  Clearly,  this  technique  requires  considerable  skill  (not  to  mention 
restraint)  on  the  part  of  the  leader.  The  AAR  leader  should  almost  always 
know  the  answer  to  the  question  he  is  asking.  Indeed,  if  he  does  not  have  a 
fairly  accurate  idea  of  what  the  answer  to  his  question  shouM  be,  the  chances 
are  good  that  he  does  not  have  a  clear  idea  of  a  teaching  point. 

The  following  example  illustrates  the  application  of  the  AAR  questioning 
technique  (Table  A-l).  In  this  example  the  trainer  is  leading  a  platoon  AAR 
and  has  covered  key  events  up  to  initial  contact.  Suppose  the  AAR  leader  was 
aware  that  one  of  the  platoon’s  squads  had  tried  to  engage  OPFOR  vehicles  with 
VIPERS  beyond  their  maximum  effective  range.  This  is  how  the  AAR  leader  might 
guide  the  discussion  of  the  teaching  point. 


Table  A-l 


Sample  of  AAR  Questioning  Technique 


Comments 


AAR  Dialog 


ue 


AAR  leader  starts  to 
identify  "what  happened." 


AAR  leader  asks  for 


AAR  LEADER:  WHAT  WAS  THE  FIRST  THING  YOU 
SAW? 

1ST  SQUAD  LEADER:  WELL  SIR,  WE  SAW  ONE  OF 
THE  BMPS  COME  OUT  OF  THE  WOODLINE.  I  COULD 
SEE  MY  DRAGON  GUNNER  WAS  ABOUT  TO  FIRE  HIM  UP 
WHEN,  ALL  OF  A  SUDDEN,  A  SECOND  BMP  CAME  OUT 
RIGHT  ON  THE  FIRST  ONE'S  TAIL. 

AAR  LEADER:  THEN  WHAT  HAPPENED? 


more  detail. 


Participant  relates  his 
plan. 


AAR  leader  begins  to 
isolate  error. 


1ST  SQUAD  LEADER:  WELL,  I  FIGURED  THAT  IF  WE 
GOT  THE  TRAIL  BMP  FIRST  WE’D  TRAP  THE  LEAD 
BMP  BECAUSE  HE  WOULDN’T  HAVE  ROOM  TO  BACK  UP. 
THEY  WERE  OUT  OF  RANGE  FOR  EVERYTHING  EXCEPT 
THE  DRAGON  AND  THE  60. 

AAR  LEADER:  GOOD  THINKING,  BUT  WHAT  HAPPENED? 


Participant  has  iden¬ 
tified  a  probable  error. 

AAR  leader  enlarges  scope 
of  discussion  by  involving 
key  participants  in  the 
discussion. 


1ST  SQUAD  LEADER:  WELL  SIR,  TWO  VIPER  GUNNERS 
GOT  NERVOUS  AND  FIGURED  THEY  COULDN'T  PASS 
UP  SUCH  A  GOOD  TARGET. 

AAR  LEADER:  OK,  HOLD  ON  A  MINUTE— VIPER 
GUNNERS,  WHERE  ARE  YOU?— WHAT  HAPPENED? 

1ST  VIPER  GUNNER:  WE  FIRED  BUT  DIDN'T  GET  ANY 
HITS. 
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Table  A-l  (continued) 


Comments 

AAR  Dialogue 

AAR  leader  attempts  to 

have  participant  diagnose 

the  error.  This  is  "Why 

it  happened?" 

AAR  LEADER:  DO  YOU  KNOW  WHY? 

Participant  diagnoses 

1ST  VIPER  GUNNER:  WELL  SIR— THEY  WERE  OUT  OF 

' 

error. 

RANGE.  AFTER  EVERYTHING  WAS  ALL  OVER,  WE 

LOOKED  AT  A  MAP  AND  THEY  WERE  AT  LEAST  400 

METERS  AWAY.  I  GUESS  WE  JUST  GOT  EXCITED 

SEEING  THOSE  TRACKS. 

AAR  leader  tries  to  get 

participant  to  identify 

another  error. 

AAR  LEADER:  WHAT  ELSE  DID  YOU  LEARN? 

2D  VIPER  GUNNER:  WELL  SIR,  AFTER  THE  SQUAD 

LEADER  GAVE  US  A  COUNSELING  SESSION  WE  FOUND 

OUT  WE  WEREN'T  SUPPOSED  TO  FIRE  'TILL  HE  TOLD 

US  TO.  HE  SURE  MADE  THAT  CLEAR. 

AAR  leader  starts  to 

AAR  LEADER:  SQUAD  LEADER,  HOW  COULD  YOU  HAVE 

explore  alternatives. 

CONTROLLED  THEIR  FIRES? 

Participant  gives  one 

1ST  SQUAD  LEADER:  HOW  'BOUT  HAND  OR  ARM 

alternative. 

SIGNALS  SIR? 

AAR  leader  presses  group 

AAR  LEADER:  YEAH,  THAT’S  ONE  WAY,  CAN  YOU 

for  another  alternative. 

Fosters  group  problem 

THINK  OF  ANOTHER? 

solving . 

1ST  SQUAD  LEADER:  AH— NOT  RIGHT  NOW  SIR. 
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Table  A-l  (continued) 


Comments 

AAR  Dialogue 

AAR  leader  involves 

AAR  LEADER:  ANYBODY  ELSE  GOT  ANY  IDEAS? 

more  participants. 

Participant  notes  another 

SOLDIER  FROM  2D  SQUAD:  SIR— HOW  ABOUT  FIGURING 

alternative.  “How  can 

OUT  WHERE  THE  MAX  RANGE  IS  AHEAD  OF  TIME  AND 

we  do  it  better?" 

SAYING  ANYTHING  CLOSER  THAN  THAT  SHOULD  BE 

FIRED  UP. 

AAR  LEADER:  DO  I  HEAR  YOU  SAYING  YOU  WOULD 

MAKE  RANGE  CARDS? 

SOLDIER  FROM  2D  SQUAD:  YES  SIR. 

AAR  leader  has  the  squad 

AAR  LEADER:  OK  SQUAD  LEADER,  CAN  YOU  TELL  US 

leader  summarize  the 

WHAT  WE  LEARNED  ABOUT  FIRE  CONTROL? 

discussion  and  restate 

the  teaching  points. 

SQUAD  LEADER:  YES  SIR.  FIRE  DISCIPLINE  IS 

VERY  IMPORTANT  AND  YOU  DON’T  WANT  TO  GIVE 

AWAY  YOUR  POSITIONS  BECAUSE  OF  A  SIGNATURE 

IF  YOU  CAN'T  GET  A  KILL.  I’VE  GOTTA  MAKE 

SURE  THAT  MY  SQUAD  HAS  A  FIRE  CONTROL  SOP 

1 

1 

AND  THAT  EVERYONE  UNDERSTANDS  THE  PROCEDURES. 
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The  questioning  technique  in  the  example  is  equally  applicable  at  squad, 
platoon,  company  and  battalion  levels.  The  AAR  leader  first  has  participants 
define  the  situation,  then  identify  its  causes,  and  finally  explore  how 
performance  could  have  been  improved. 

The  timing  of  the  AAR  is  important.  The  AAR  should  be  conducted  as  soon 
as  possible  after  the  exercise.  If  delayed,  controllers,  leaders  and  troops 
will  tend  to  forget  the  details  of  engagements,  critical  events,  FRAGOs, 
spatial  relationships,  etc.  The  more  time  and  events  between  the  end  of  the 
exercise  and  the  AAR,  the  more  will  be  forgotten,  and  the  less  useful  the  AAR 
will  be.  An  AAR  can  be  delayed  a  few  hours  if  necessary  with  little  adverse 
effect;  but  if  delayed  a  day  or  more,  the  AAR  may  be  of  little  value.  By  that 
time  many  details  will  have  been  forgotten  or  confused  with  other  events.  The 
trainer  should  conduct  the  AAR  while  the  exercise  is  still  fresh  in  everyone's 
minds.  Another  situation  which  should  be  avoided  is  to  conduct  two  (or  more) 
exercises  followed  by  comprehensive  AAR.  Experience  has  shown  clearly,  that 
events  in  the  two  exercises  tend  to  become  confused,  making  the  AAR  both 
difficult  to  organize  and  conduct,  and  not  very  beneficial.  The  two 
principles  of  AAR  timing  are:  (a)  conduct  the  AAR  as  soon  ac  possible  after 
the  exercise,  and  (b)  conduct  an  AAR  for  each  exercise  separately. 

Visual  aids  should  be  used  in  the  AAR.  They  help  everyone  to  picture 
the  terrain  and  tactical  situation,  and  they  increase  learning.  The  kinds  of 
aids  that  are.,  desirable  depend  on  the  echelon,  and  on  the  location  of  the 
AAR.  In  most  cases,  AARs  should  be  conducted  where  all  participants  have  a 
view  of  the  terrain  on  which  most  of  the  action  occurred.  Usually,  the 
defensive  position  or  objective  provides  a  good  location,  especially  if  the 
route  of  advance  of  the  attacking  unit  can  be  seen. 
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Squad  and  platoon  AARs  are  most  often  held  in  the  field.  At  the  most 
simple  level,  the  AAR  leader  can  sketch  the  necessary  information  on  the 
ground,  using  sticks  and  stones  to  indicate  weapon  systems,  units,  objectives 
and  so  on.  This  is  particularly  good  for  squad  exercises  and  may  be  improved 
upon  by  using  miniature  weapon  system  models  available  in  many  toy  and  variety 
stores.  Another  alternative  is  to  use  a  tripod-  mounted  briefing  chart  and 
felt  pens.  This  has  the  advantage  of  being  more  visible  to  the  participants 
than  is  the  "ground  sketch.” 

For  company  and  higher  level  AARs  conducted  in  the  field,  the  briefing 
chart  approach  is  probably  best.  Also,  the  terrain  can  be  sketched  on  target 
cloth  prior  to  the  exercise,  and  later  natural  objects  can  be  used  to  indicate 
vehicles,  objectives,  etc.  If  it  is  absolutely  necessary,  AARs  can  be 
conducted  in  garrison.  Some  posts  have  scaled  terrain  models  of  training 
areas  often  equipped  with  various  weapons  system  models.  These  are  generally 
excellent  for  AARs.  Another  alternative  is  a  standard  classroom  or  meeting 
room  which  usually  come  equipped  with  backboards. 

Finally,  the  soldiers  and  other  AAR  participants  should  be  asked  to 
indicate  their  positions  or  routes,  rather  than  having  it  done  by  the  AAR 
leader.  This  increases  the  sense  of  participation  and  eliminates  possible 
misinterpretation  of  the  soldiers’  comments. 
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PERFORMANCE  EVALUATION  AND  TRAINING  DIAGNOSIS 
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APPENDIX  B 


PERFORMANCE  EVALUATION  AND  TRAINING  DIAGNOSIS 

This  appendix  presents  some  concepts  underlying  unit  performance 
evaluation,  and  general  method  for  identifying  performance  problems.  It  is 
hoped  that  by  presenting  concepts  and  methods  together,  a  m^re  complete 
picture  can  be  drawn  of  the  relationship  between  evaluation  strategy  and  the 
development  of  unit  proficiency  than  was  possible  in  the  procedural  sections 
of  this  document. 

EVALUATION 

There  are  a  wide  variety  of  possible  reasons  for  evaluating  unit 
performance.  Evaluations  may  be  conducted  to  assess  the  training  components 
of  force  readiness,  to  estimate  the  impact  of  major  command  and  local 
policies,  to  identify  requirements  for  training  methods  or  devices,  to 
identify  needs  for  improved  training  doctrine,  to  assist  in  training  program 
development,  and  to  identify  immediate  unit  training  needs.  Regardless  of 
whether  we  are  concerned  with  broad  policy  issues  or  with  local  training 
needs,  answers  to  two  fundamental  questions  are  required:  what  is  the  current 
status?  what  is  the  best  course  of  action? 

The  answer  to  the  first  question  is  a  direct  result  of  evaluation. 
Evaluations  yield  descriptive  information  about  characteristics  and  qualities 
of  whatever  is  being  evaluated.  Finding  answers  to  the  second  question,  "what 
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is  the  best,  course  of  action?"  is  a  more  complicated  matter.  Information 
about  current  status  has  to  be  combined  with  information  about  goals,  costs, 
estimates  of  potential  benefits,  priorities,  possible  undesirable  side- 
effects,  etc.  It  is  largely  because  of  these  situation-specific  complications 
that  most  evaluation  guidance  tends  to  treat  assessment  of  current  status  in 
more  depth  than  issues  related  to  future  courses  of  action.  While  the 
principal  thrust  here  has  been  on  the  evaluation  processes  supporting  current 
status  assessments,  there  has  also  been  seme  emphasis  on  training  diagnosis 
which  ^itovides  a  bridge  between  the  description  of  unit  performance 
characteristics  and  the  determination  of  options  for  further  training. 

Training  diagnosis  denotes  the  process  of  taking  descriptive  "current 
status"  information  and  making  inferences  abouL  the  causes  of  the  actions  or 
results  described.  That  is,  training  diagnosis  is  finding  out  why  things 
happen.  To  use  a  medical  analogy,  a  patient  sees  a  docLor  and  describes  his 
condition.  The  doctor  may  also  perform  some  laboratory  tests.  These  provide 
the  doctor  with  information  on  the  patient's  current  status.  The  physician 
uses  this  information  to  determine  the  causes  of  the  patient's  condition.  His 
determination  is  the  diagnosis.  The  diagnosis  together  with  other  information 
about  the  patient  (past  history,  sensitivity  to  particular  drugs  etc.)  is  then 
used  to  prescribe  a  treatment.  It  is  common  in  this  context  for  patients  with 
identical  diagnoses  to  receive  different  treatments.  The  same  general  model 
can  bo  used  to  describe  the  evaluation  process.  The  evaluation  team  produces 
descriptive  information  about  the  unit's  performance.  The  evaluator  uses  that 
information  to  determine  the  reasons  for  actions  and  results  described.  The 
unit  commander  then  uses  that  diagnosis  along  with  other  information  about  the 
unit,  resources  available,  etc.  to  formulate  a  remedy  for  the  unit's 
performance  problems.  This  is  the  basic  evaluation  model:  description  forms 
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the  basis  for  diagnosis  and  diagnosis  is  a  key  ingredient  for  determining  the 
needs  for  further  training. 

In  the  preceding  sections  of  Lhe  document  the  primary  focus  was  on  the 

procedures  for  producing  accurate  descriptions  of  performance.  Though  not 

stated  explicitly,  the  descriptive  information  consists  of  two  basic  types: 
information  about  unit  processes  and  information  about  outcomes.  By  far  the 
most  frequently  encountered  type  is  process  information.  Outcome  information, 
like  casualties,  mission  accomplishment,  etc.,  is  largely  excluded  from  the 
ARTEP  T&EOs,  and  similar  training  documents.  There  are  two  principal  reasoi  i 
for  this  exclusion.  First  outcome  information  is  more  easily  contaminated  by 
factors  unrelated  to  the  unit's  performance.  Characteristics  of  Lhe  OPFOR, 

environmental  factors,  terrain,  force  ratios,  etc.  may  determine  outcome  as 

much  as  the  proficiency  of  the  unit.  A  second  more  important  reason  is  that 
outcome  measures,  by  themselves,  give  few  clues  about  causes.  A  unit  may  not 
accomplish  its  mission  or  the  exercise  may  result  in  a  poor  casualty  exchange 
ratio,  but  that  information  is  inadequate  for  training  diagnosis  and 

development  of  subsequent  training  plans.  There  are,  however,  uses  for 

outcome  information.  They  can  serve  as  flags  which  signal  probable 

performance  problems  and  cue  the  evaluator  to  examine  certain  aspects  of  Lhe 
exercise  more  closely.  Outcome  information,  when  used  in  conjunction  with 

process  information,  can  suggest  how  serious  performance  problems  may  be,  and 
may  therefore  be  useful  in  setting  future  training  priorities. 

ARTEP  T&EO  standards  consist  almost  totally  of  process  measures.  The 
performance  standards  are  based  on  those  unit  actions  which  are  believed  to 
result  in  the  best  possible  chance  of  success  on  the  battlefield.  Moreover, 
process  measures  provide  the  best  basis  for  diagnosing  training  deficiencies 
and  have  the  advantage  of  being  relatively  independent  of  OPFOR  actions  and 


55 


other  variables  which  often  affect  outcome  measures.  The  major  problem 
associated  with  these  measures  is  their  number.  Because  the  T&EOs  have  been 
designed  to  pinpoint  very  specific  problems  in  highly  complex,  variable 
situations,  there  are  usually  an  awkwardly  large  number  of  standards  within  an 
evaluation  exercise.  By  using  outcome  information  (primarily  casualty  data) 
to  flag  potential  unit  performance  problems,  the  difficulties  of  dealing  with 
large  numbers  of  process-oriented  standards  can  be  much  alleviated.  The 
following  section  deals  with  a  genera)  method  for  training  diagnosis  and 
suggests  how  process  and  outcome  information  can  be  used  to  identify 
performance  problems. 

TRAINING  DIAGNOSIS 

Training  diagnosis  is  an  art — there  are  no  absolute  rules  to  guide  the 
evaluator.  Yet,  there  are  some  general  principles  that  can  help  the  evaluator 
structure  his  inquiry  into  the  Mwhys“  of  tactical  performance.  The  evaluator 
is  a  detective  and  a  large  part  of  his  activity  is  concerned  with  finding  out 
why  important  events  occurred.  The  first  requirement  is  to  sorL  out  what  is 
important  from  what  is  not.  Unfortunately,  much  of  what  is  important  only 
becomes  apparent  long  after  the  events  have  occurred.  For  that  reason,  the 
evaluator  needs  to  become  an  expert  at  tracing  chains  of  events  back  to  their 
sources.  One  event  will  cause  another  which  will  in  turn  cause  another  and  so 
on.  Frequently,  several  such  chains  of  events  come  together  to  influence  the 
outcome  at  some  critical  point  in  the  exercise.  Being  able  to  trace  these 
kinds  of  chains  of  events  is  a  key  to  the  art  of  diagnosis. 

The  evaluator's  detective  work  can  be  broken  down  into  several 
sequential  steps:  the  evaluator  first  determines  what  happened,  then  how  it 
happened,  and  finally  why  it  happened.  Also,  the  evaluator  should  make 
assessments  of  the  unit's  tactical  options;  that  is,  what  could  have  been  done 
differently  to  improve  the  outcome  of  the  event  or  exercise. 
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These  assessments  are  usually  made  during  the  course  of  the  MILES 
controller/evaluator  debriefing  held  just  before  the  After  Action  Review. 
During  the  controller/evaluator  debriefing  the  evaluator  receives  information 
from  his  subordinate  team  members.  Clearly  the  higher  the  echelon  being 
trained,  the  more  the  evaluator  will  have  to  depend  on  other  controllers  for 
reliable  information  on  the  exercise,  and  for  evaluations  of  subordinate  unit 
performance.  Also,  other  factors  such  as  type  of  unit  being  trained,  terrain, 
mission,  etc.  will  affect  level  of  detail  covered  in  controller  debriefing. 
For  example,  in  a  dismounted  rifle  squad  hasty  attack,  an  evaluator  can  often 
see  and  hear  most  of  the  action  in  the  exercise.  Therefore,  the  controller 
debriefing  can  be  rather  short,  primarily  focusing  on  filling  in  and 
confirming  details.  In  contrast,  consider  a  full,  combined  arms,  company-team 
delay  mission.  The  size  of  the  unit,  the  amount  of  terrain  involved,  the 
complexity  of  the  required  maneuvers,  etc.  will  combine  to  make  an  extensive, 
detailed  controller  debriefing  necessary.  Usually,  the  senior  evaluator  will 
only  be  able  to  observe  a  part  of  the  action.  Other  evaluators  (and  often  key 
OPFOR  participants)  must  supply  the  information  necessary  to  determine  the 
reasons  why  key  events  occurred.  It  cannot  be  emphasized  too  strongly:  the 
purpose  of  the  MILES  controller/evaluator  debriefing  is  to  provide  the  senior 
evaluator  (AAR  leader)  with  accurate,  detailed  information  on  not  only  what 
happened  but  also  how  and  why  events  occurred,  and  most  importantly  on  what 
could  have  done  differently  to  improve  outcomes.  Each  step  of  the  training 
diagnosis  process  is  discussed  in  the  following  paragraphs. 

What  Happened.  The  evaluator’s  first  job  is  to  select  an  important  event  for 
analysis.  Important  events  in  MILES  exercises  are  most  often  associated  in 
one  way  or  another  with  casualties;  the  more  casualties  a  unit  inflicts  or 
sustains,  the  more  important  that  event  is  likely  to  be.  The  importance  of 
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casually-related  events  depends  on  the  echelon  in  question.  For  a  platoon, 
the  loss  of  an  APC  is  likely  to  be  important.  But,  at  the  company  level,  such 
a  loss  is  likely  to  be  of  lesser  importance. 

There  are  three  major  reasons  why  casualty  events  are  likely  to  be  good 
starting  points  for  the  evalutor's  detective  work.  First,  they  are  often  the 
end  of  a  series  of  actions  that  were  unusually  well  or  unusually  poorly 
done.  Second,  casualties  inflicted  or  sustained  often  have  a  bearing  on 
mission  outcome  because  they  alter  the  relative  firepower  available  to  the  two 
forces.  Finally,  casualties  are  clearly  understood  common  denominators  of 
warfare.  Every  commander  or  leader  wishes  to  maximize  casualties  inflicted 
while  minimizing  those  sustained.  This  orientation  will  provide  a  basis  for 
discussion  and  understanding  during  the  After  Action  Review. 

Naturally,  other  types  of  events  may  be  selected  as  important  even 
though  they  may  not  result  in  casualties  inflicted  or  sustained.  A  unit  may, 
for  example,  be  responsible  for  a  major  security  breach  which  goes  undetected 
or  is  not  taken  advantage  of  by  the  enemy.  Another  example  would  be  a  unit’s 
failure  to  provide  good  indirect  fire  support  for  its  subordinate  elements, 
but,  because  of  an  outstanding  performance  by  its  smaller  units,  the  unit  may 
achieve  an  overwhelming  victory.  There  are  a  great  many  events  that  do  not 
result  in  casualties  but  are  nonetheless  important.  On  the  whole,  however, 
the  evaluator  will  find  that  casualty-related  events  generally  provide  the 
best  ground  for  meaningful  diagnosis. 

Having  selected  an  important  event,  the  evaluator's  next  job  is  to 
define  the  event's  characteristics.  The  evaluator  should  seek  information  on 
the  identities  of  the  element(s)  involved,  and  the  time  and  location  of  the 
event.  Most  of  this  is  relatively  simple  for  casualty-related  events.  The 
relevant  information  is  usually  available  from  element's  controller/evaluator 
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or  from  opposing  force  (OPFOR).  Evaluators  should  be  encouraged  strongly  to 
make  written  notes  during  the  exercise,  as  this  will  greatly  help  in 
reconstructing  the  sequence  of  important  events  during  the  controller 
debriefing. 

How  it  Happened.  It  is  during  this  step  that  the  evaluator's  true  detective 
work  begins.  Having  determined  what  happened,  the  evaluator  now  tries  to 
increase  his  understanding  by  gathering  facts  about  actions  preceding  and 
following  the  event.  He  must  develop  a  relatively  complete  understanding  of 
both  the  event  in  question  as  well  as  closely  related  actions  and  events.  For 
a  casualty  event,  the  evaluator  would  try  to  find  out  what  the  casualties 
(i.e.,  targets)  were  doing  just  prior  to  being  engaged,  what  adjacent  elements 
were  doing,  how  the  targets  were  acquired,  etc.  Most  of  this  kind  of 
information  will  have  to  be  obtained  from  the  other  evaluators  and  from  the 
OPFOR. 

The  key  to  this  step  is  the  evaluator's  ability  to  ask  tne  righL 
questions.  At  the  lower  echelons,  the  right  questions  are  most  frequently 
related  to  what  a  given  unit  did,  that  is,  to  execution.  But  at  higher 
echelons,  important  questions  are  more  often  related  to  what  command  elements 
knew  about  the  situation  and  what  decisions  they  made.  For  example,  suppose 
that  a  lead  company  is  moving  forward  when  it  is  engaged  by  the  OPFOR  who  pins 
down  two  of  the  company's  platoons.  Suppose  also  that  the  third  platoon  was 
not  close  enough  to  the  OPFOR  to  deliver  effective  fire.  At  the  lower  echelon 
(platoon),  the  evaluator  will  be  primarily  interested  in  questions  related  to 
platoon  fire  and  maneuver:  How  did  the  engagement  begin?  What  were  the 

platoons'  reactions  to  receipt  of  fire?  Did  platoon  leaders  reporL  the 

i 

engagement?  Was  the  available  cover  used  effectively?  Did  platoons  return 
OPFOR  fire  as  effectively  as  possible?  Etc.  Data  from  ARTEP  T&EOs  can  often 
be  used  to  answer  many  such  questions. 
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At  the  company  level,  the  evaluator  would  need  to  ask  different  types  of 
questions:  Did  the  commander  realize  that  two  of  his  platoons  had  become 

heavily  engaged?  Did  he  have  accurate  information  on  all  platoon  locations? 
Did  he  attempt  to  get  information  on  OPFOR  locations  and  strength?  What 
decision  did  he  make  about  moving  the  third  platoon  into  a  position  where  it 
could  provide  support  to  the  two  which  were  pinned  down?  Did  he  request 
indirect  fire  support?  Etc. 

In  summary,  the  how-it-happened  step  is  geared  toward  gathering  as  many 
facts  as  possible  about  important  exercise  events.  Exactly  whaL  facts  should 
be  gathered  depends  on  echelon,  mission,  scenario,  disposition  of  forces, 
friendly  and  enemy  situations,  etc.  As  noted  earlier,  many  of  the  important 
facts  will  not  be  obvious:  very  careful  debriefing  of  the  evaluators  and 
sometimes  of  the  OPFOR  will  be  necessary  to  get  the  needed  information. 

Why  it  Happened:  This  is  the  final  and  perhaps  most  difficult  step  of  Lhe 
diagnostic  process.  Here  the  evaluator’s  job  is  to  organize  the  facts  he  has 
gathered  and  make  inferences  about  the  causes  of  the  events  in  question.  He 
must  bring  his  tactical  expertise,  analytic  ability,  and  frequently  a 
considerable  amount  of  intuition  to  bear  on  the  problem  of  finding  the 
fundamental  causes  of  events  he  has  chosen  to  analyze. 

The  evaluator  first  needs  to  organize  the  factr  related  to  the  event  of 
interest.  Key  words  and  phrases  indicating  relevant  actions  and  events  suould 
be  listed  in  their  order  of  occurrence.  It  is  al  j  useful  to  indicate  the 
approximate  time  of  the  event.  Most  often,  some  of  these  events  will  be  prior 
to  the  one  of  interest  while  others  will  occur  later.  Next,  prioritize  the 
events  which  occurred  before  the  one  in  question  in  terras  of  how  much  each 
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preceeding  event  contributed  to  causing  the  latter  one.  This  process  will 
help  to  identify  the  immediate  causes.  However,  more  depth  will  usually  be 
required  to  determine  the  more  basic  causes.  Therefore  the  evaluator  should 
take  the  one  or  two  highest  priority  immediate  causes  and  repeat  the 
prioritization  process  in  order  to  identify  secondary  or  earlier  causal 
events.  A  simplified  diagram  of  this  process  is  shown  in  figure  B-l. 
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Figure  B-l:  Determining  a  Causal  r-.iin  of  Events 

The  procedure  should  be  repeated  until  the  evaluator  is  confident  he  has  found 
the  basic  causes  of  the  critical  event.  Occasionally  the  immediate  cause  will 
also  be  the  basic  cause  of  a  critical  event.  But  frequently  the  basic  cause 
will  be  two  or  three  times  removed  from  the  critical  event.  There  are  a 
couple  of  guidelines  which  can  help  the  evaluator  estimate  when  his  analysis 
has  reached  back  far  enough  to  reveal  the  "basic”  cause  of  a  chain  of 
events.  First,  the  next  level  of  causal  events  will  usually  be  difficult  to 
prioritize.  When  several  proceeding  events  seem  to  have  similar  effects  in 
causing  the  next  event,  the  event  is  probably  a  "basic"  one.  Second,  "basic" 
causal  events  will  often  reveal  a  commonly  encountered  performance  problem. 
For  companies  and  larger  units  these  problems  will  frequently  be  related  to 
information  transfer  and  decision  processes.  For  smaller  units  most  problems 
will  concern  execution. 

Finally,  once  the  basic  causes  are  isolated,  it  will  be  necessary  to 
determine  how  best  to  eliminate  the  problem.  Some  problems  can  be  remedied  by 
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using  good  After  Action  Review  techniques  followed  by  additional  exercises. 
In  this  case,  the  evaluator  should  pay  particular  attention  to  problems 
discovered  in  the  course  of  earlier  exercises.  Other  types  of  problems  may 
require  different  approaches  both  because  of  teaching  effectiveness 
considerations  and  because  of  resource  constraints.  For  example,  often 
command  and  control  can  be  better  improved  using  TEWTS,  CPXs,  etc.  than  by 
using  full-unit  MILES  exercises.  It  should  be  remembered  that  even  the  best 
training  diagnosis  is  not  worthwhile  unless  sound  remedies  for  the  problems 
uncovered  are  used. 
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